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Repeated  Measurements  on  Autoregressive  Processes 

by 

T.  W.  Anderson 
Stanford  University 

1,  Introduction. 

Time  series  analysis  is  developed  mainly  for  the  study  of  a single 
sequence  of  observations.  In  this  paper  methods  of  statistical  in- 
ference are  developed  for  a set  of  several  sequences  of  observations 
from  the  same  autoregressive  vector-valued  process.  Having  several 
time  series  available  from  the  same  model  permits  investigation  of 
such  questions  as  whether  the  process  is  homogeneous,  that  is,  whether 
the  autoregression  coefficients  are  constant  in  time;  if  the  coeffi- 
cients vary  in  time  they  can,  nevertheless,  be  estimated  from  several 
series  even  though  there  may  be  no  pattern  to  the  variation. 

The  distributions  of  estimates  and  test  criteria  in  time  series 
analysis  are  usually  asymptotic  as  the  length  of  the  series  increases. 
In  the  present  case  of  repeated  measurements,  asymptotic  theory  can 
alternatively  be  based  on  the  number  of  observed  series  increasing. 

In  cases  where  both  iarge-sample  theories  might  be  used  they  are 
compatible. 
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There  are  many  applications  of  these  procedures  of  importance. 

In  a panel  survey  (in  which  several  respondents  are  interviewed  at 
more  than  one  point  in  time)  the  responses  may  be  quantitative,  such 
as  answers  to  the  question  how  many  hours  did  you  spend  last  month 
reading  the  newspapers.  In  economic  surveys  the  questions  are  likely 
to  produce  numerical  answers:  how  many  hours  did  you  work  last  week 

and  how  much  money  did  you  spend  on  groceries  last  month.  Analysis 
of  such  data  are  sometimes  called  cross-section  studies  by  econome- 
tricians, A psychologist  may  obtain  a test  score  on  several  indivi- 
duals at  several  dates;  a physician  may  read  blood  pressures  of  his 
patients  on  several  days. 

The  purpose  of  the  present  paper  is  to  develop  statistical  methods 
for  problems  of  autoregressive  processes  in  the  case  of  repeated  mea- 
surements which  are  analogs  of  procedures  for  corresponding  problems 
of  Markov  chains  treated  by  Anderson  and  Goodman  (1957).  Anderson 
(1976)  reviewed  some  of  these  procedures  for  Markov  chains  and  sug- 
gested analogs  for  autoregressive  processes.  This  present  paper  de- 
velops those  and  other  procedures  in  greater  detail  and  more  generally 
and  provides  Justification  for  some  assertions  in  the  earlier  paper. 
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2 > The  Model « 

If  y is  a p-component  vector*  a Markov  (first-order)  vector 
process  with  mean  = 0 Is  defined  by 

It  - ^ 'it  • 

where  B(t)  is  a pxp  matrix  and  ' . } is  a sequence  of  independent 

*v 

(unobservable)  random  vectors  with  expected  values  ^u^  = 0 , covari- 
ance matrices  j independent  of  i»  yt  

Let  the  covariance  matrix  of  y^  be  ^y^y^  = . Then  from  (2.1) 

and  the  independence  of  y^  ^ and  u^  we  deduce 

(2.2)  = B(t)  B(t)’  + . 


If  the  observations  are  made  for  t = 1,  ...»  T,  the  model  may  be 
specified  by  the  marginal  distribution  of  y^  and  the  distributions 
of  u^,  ....  U.J,  . In  particular,  if  and  the  are  normal, 

the  model  for  the  observation  period  is  specified  by^ 

<^^7^2) B(T),  •••.  ?T  • 

When  the  autoregression  matrices  are  homogeneous,  that  is, 

B(t)  = B , and  the  u 's  are  identically  distributed  with  mean  0 
and  covariauice  matrice  Z , the  process  is  stationary  if  the  charac- 
teristic roots  of  B are  less  than  1 in  absolute  value  and  the  process 
is  defined  for  t = . . . , -1,  0,  1 , . . . or  if  y^  is  assigned  the  sta- 
tionary marginal  distribution.  In  this  case  the  covariance  matrix  of 
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A higher-order  autoregressive  process  may  be  written  as  a first- 
order  process  by  suitable  redefinition.  For  example,  a second-order 
autoregressive  vector  process  with  homogeneous  autoregressive  coeffi- 
cients defined  by 


(2.8) 


it  = !i  ?t-i 


!2  lt-2  * 'tt 


can  be  written  as  a first-order  process  by 

(2-9)  = ! it-1  5t  ’ 


where 

(2.10) 

(2.11) 


lit 
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The  characteristic  roots  of  B are  the  roots  of 


(2.12)  I -1^1  + + B^l  = 0 . 

i! 

f;  For  a stationary  process  these  roots  are  to  be  less  than  1 In  absolute 

value . 

i 
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The  autoregressive  processes  appi'opriate  to  several  subpopulations 


i 
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(strata)  may  be  different.  In  the  inhomogeneous  first-order  case  the 
matrices  in  the  h-th  process  maj'  be  B^'^^(t),  h = i,  s ; in 

the  homogeneous  case  they  may  be  h - 1,  s . If  in- 

fluencing variables  are  continuous,  they  may  be  taken  account  of  by 
adding  them  to  the  regression  to  yield  the  mode],  (in  t’ne  first-order 
homogeneous  case) 

^t  = ? ^t-1  • 

where  is  a vector  of  such  variables  and  Y is  a vector  of  para- 

meters. In  particular,  when  z = 1 and  Y is  a scalar,  the  process 
{y^}  may  have  a mean  different  from  0 . 

The  autnregres: ive  process  is  constructed  from  ( ccmdi tional ) raul- 
fivariatt-  reg'^essions . In  (B.l),  for  example,  the  vc-tor  con- 
st Itutes  the  ''independent  variat'les"  and  the  vec+or  const:  tutes 

t.'.e  "iependent  vaxiai’es"  in  ordinary  regression.  'I  a las'ge  e;;tenL 
f.’ie  stat' s.tical  metli'bs  f-r  autoregressive  mo>dels  are  regression  or 
' east  ouuai'es  procedures. 
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3.  Estimation  of  Autore'^ressive  Coefficients. 


Let  y.  be  the  p-component  vector  of  measurements  of  the  a-th 


individual  at  the  t-th  time  point,  a = l,  t-1,  T . The 

model  is  a first-order  autoregressive  model  (2.1)  with  u^  having  the 

normal  distribution  N(0,Z  ) and  y having  the  normal  distribution 

u -wX 

N(0,F^)  . The  probability  density  of  the  sequences  y^^,  y^^, 

a = 1,  . . . , N,  is 


N 


(3.1)  n 


a=l  kp  ^ T p 

(2tt)^  If  n Is 

t=2 


^Tn  ^ T |w 


exp 


;rHTp  T 

(27t)2  ]f^|2  n |E^| 


la 


t=2 


t=2 


N 


1 i 5“  expX  i-  tr  F'^  i y y^ 

^p  ^ T ^ 1 ‘"L  ^ a^l  ^ 

{2.f  ]fJ2  n Is 

t=2 


T W N , N 

J,  "■  5t  ( I ?ta  it<i  - ^ It 

i=2  'a=l  a=l 


to 


0=1  ’ * 


1 
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Then  l/N  times  the  logarithm  of  the  likelihood  function  is 


T 

(3.2)  I log  L = - |Tplog  2tt-|  log  IfJ  -i-  \ log  | 1 - | ■ 

t “2 


tr  F~^C^ (0) 


n — T • ’ 

+ [ tr  tC^(O)  - 2 C^(l)  B (t)  + B(t)  C^_]^(0)  B (t)] 


where 


(3.3) 


^ ~ N ^ta  ?t-j  ,a 


Then  a sufficient  set  of  statistics  for  ,B( 2) , . . . ,B(T) , . . . ,E^  is 
5^(0),  t = 1,  T , and  C^(l),  t = 2,  ...,  T . 

The  derivatives  of  (l/N)log  L with  respect  to  the  elements  of 
B(t)  form  the  matrix 


(3.4) 


f Hrfr' - - 5<‘> 


The  maximum  likelihood  estimates  of  B(2),  ...,  B(t)  are 


(3.5) 


B(t)  = C^(l)  C"_^(0)  , 


t = 2.  T . 


Then  1/N  times  the  logarithm  of  the  concentrated  likelihood  function 
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1 1 1 ^ 

(3.6)  - I Tp  log  2tt  - i log  [f^I  - f I log  |E^ 


ijtr  r“^  C^(0)  + I tr  Z^^Jc^(O)  - B(t)  C^_j^(0)  B(t)j| 


Then  [by  Lemma  3.2.2  of  Anderson  (1958)]  the  maximum  likelihood  es- 
timates of  , Eg  > • • • 5 ^fj. 


(3.7) 


!l  = Sl^°^  ’ 


(3.8)  E^  = C^(0)  - B(t)  C^_i(0)  B(t) 


C^(0)  - C^(l)  c^.i(o)-^  C^(l)’ 


N \ ’ 

I Jl(?ta"  !^"^Vl,a)(yta-  ^‘It-l ,a)  ’ 


t=2,  ....  T 


The  components  of  B(t)  are  least  squares  estimates  and  E are 

-w  Xl 

composed  of  sums  of  products  of  deviations  from  the  fitted  (auto)  re- 
gressions. [See  Anderson  (1958),  Chapter  8,  and  Anderson  (1971), 
Chapter  5,  for  example.] 

The  assumption  chat  the  autoregression  matrices  aire  homogeneous 
and  the  disturbances  identically  distributed  leads  to  considerable 
simplification.  Then  l/N  times  the  logarithm  of  the  likelihood 
function  is 
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(3.9)  I log  L » - I Tplog  2tt  - I log  |f^|  - |{T-1)  log  \Z\ 


- ijtr  C^(0)  + tr  ^ f 


- 2 I C (1)  B + B 


!']) 

- Tp  log  2ir  - log  |F^|  - |{T-1)  log  |e1 


C^(0)  + (T-l)  tr  ^C(O)  - 2 C(l)  B* 


+ B C*(0)  b'J|  , 

where 

(3.10) 

C(J)  * ^ I c (J)  , 
^ ^ t»2 

(3.11) 

^ jo 

= c(o)  + ^ [c^(0)  - c^(o)] 


Then  a sufficient  set  of  statistics  for  F^,  B,  and  E is 

C^(0),  C(0),  C (O)  and  C(l)  . The  maximum  likelihood  estimate  of  F^ 

is  (3.7),  and  the  maximum  likelihood  estimates  of  B and  E are 
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(3.12) 


(3.13) 


B = C(l)  C*(0)~^  , 


Z = C(0)  - B C*(0)  B 


C(0)  - C(l)  C*(0)~^  C(l)’ 


T N 


c,  CX“  X 


An  alternative  model  is  to  consider  y,  , a = 1 , . . . , N , as 

11a 

nonstochastic  or  fixed  and  treat  y , t = 2,  ...,  T,  a = 1,  ...,  N , 
conditionally.  Then  the  maximum  likelihood  estimates  of  B(t)  and  , 
t = ?.,  T,  are  (3.5)  and  (3.8)  and  of  B and  Z are  (3.12)  and 

(3.13),  as  tne  may  be. 

'.'nen  y,  is  considered  to  have  the  marginal  normal  distribution 

- i.Cl 

determined  by  the  stationai'y  process,  the  covariance  matrix  is  a 

function  of  B and  Z as  given  by  (2.3).  Then  the  maximum  likelihood 
estimates  are  much  more  complicated.  In  the  simplest  case  of  p = 1 
the  estimation  equations  involve  a cubic  [Sec.  6.11.1  of  Anderson 
(1971)];  if  P > 1 or  the  order  is  greater  than  1,  polynomial  equations 
of  higher  degree  are  involved.  [As  T , (3.12)  and  (3.13)  are 
asymptotically  equivalent  to  the  maximxim  likelihood  estimates,  but  not 
as  N oo  . ] 

Estimation  in  higher-order  processes  is  similar.  For  example,  the 
second-order  case  may  be  treated  in  the  form  (2.8)  or  (2.9).  Suppose 
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pa ) 


y-,  • •••.y  ,a  = l N,  are  considered  as  nonstochastic  or  fixed. 

ila  trot’  ’ 

Then  in  the  r-th  order  process  the  maximum  likelihood  estimates  of 
(t),J=l,  ...,r,  t=  r+1,  ....  T , are  the  solutions  to 


where 


(3.15) 


Ct(i,j) 


1 N 

i T * 

N a=i  ?t-j.a  ’ 


The  covariance  matrix 

~t 


is  estimated  by 


(3.16) 


= C^(0)  - I B^(i)  C^(i,j)  B(J) 
i,J=l 


I 


In  the  homogeneous  case  the  estimates  are 


where 


(3.18) 


C(j|r)=—  I C^(j.j) 
t=r+l 


TrJ 


T-r  I * 

t=r+l-J 


(3.19) 


C(i,j|r)=;j^  I C (i,j) 
t=r+l 


The  exact  distributions  of  these  estimates  are,  in  general,  too 
complicated  to  be  useful.  Hence,  we  turn  to  asymptotic  theory.  For 
fixed  T by  the  law  of  large  numbers 


(3.20) 


Plim  C^(0)  =4^  l’t  = It  ' 

N-too 


t = 1,  ...,  T , 


:3.21) 


Plim  C^(l)  =(Jy^  y'_^  = F^ 


t = 2,  ... , T 


N-^oo 


Hence,  as  N , C (O)  is  a consistent  estimate  of  F , t = 1,  ...,  T ; 

— x« 


B(t)  is  a consistent  estiipate  of  B(t),  t=2,  ...,  T,  and  I.  is  a 
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''•-  •-<-.=  • ••>5';^'v 


consistent  estimate  of  , t = 2 , . . . , T . It  follows  from  (3.20) 
and  (3.21)  that  in  the  homogeneous  case 


(3.22) 


1 

pllm  C(0)  = — J F . 
N-ko  ^ t=2 


which  is  F in  the  stationary  case  [F^  *=  F given  by  (2.3)), 


(3.23) 


pllm  C(l)  = ^ B I F 
N-k»  ' t=2 


which  is  B F in  the  stationary  case,  and 


(3.21*) 


plim  C (0)  = I , 


N-w 


t=2 


which  is  F in  the  stationary  case.  It  follows  that  in  the  homogeneous 

^ /s 

case,  as  N -♦  » , B is  a consistent  estimate  of  B and  E is  a 
consistent  estimate  of  E . 

For  fixed  T as  N -*■  <»  the  elements  of  »^[C.  (0)  -C$C.  (0)]  , 
t = 1,  ...»  T,  and  of  »¥[C^(l)  -<5c^(l)]  have  a Joint  limiting  nor- 
m€il  distribution  with  means  0 if  it  is  assumed  that  y^  is  random 
and  the  set  ^2a’  ■’*’  '^Ta^  identically  distributed  (with 

respect  to  a = 1,  ...,  N)  . It  follows  that  *^[F^-F^] , »^[B(t )-B(t)) , 
and  y^[E^-E^]  has  a Joint  limiting  normal  distribution  with  means  0. 


It  remains  to  find  the  covariances.  The  limiting  distribution  of 


(3.25)  <tlB(t)  - B(t)l  = i I u C 

✓N  a=l  ’ 


is  the  limiting  distribution  of 


(3.26) 


1 ? • -1 

^ ^ ~ta  ?t-l,a  lt-1 
✓N  a=l 


since  ^ „ ^t  1^*^^  ~ ^t  1 ‘ covariance  matrix  of  the  i-th 

and  J-th  rows  of  (3.26)  is 


(3.27) 


1 /?  V w-1  ' w-1 

N ^ i ?t-l,a  ^t-l,Y  -t-2 

a,Y=l  ' ’ ’ 

_l^(t)  -Itff  ' -1 

■ N ij  ail  5(t-l,a  ^t-l,a  ~t-l 


- F'l 

-^ij  !t-l  ’ 


where  E = [O.V]  . The  covariances  of  the  elements  of  i/¥[B(t)-B(t)] 
~t  ‘ ij  ■’  ~ - 

in  the  limiting  distribution  constitute  the  Kronecker  product 


(3.28)  ^©rtil 
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!;-i 

^^12 

!t"-i 

...  a, 

ip 

!t'-i 

21 

f“^ 

a 

F~^ 

...  a 

F~^ 

~t-l 

22 

it-1 

2p 

lt-1 

«- 

• 

• 

pi 

F~^ 

~t-l 

"p2 

Cl 

...  a 

PP 

f”^ 

lt-1 
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The  elements  uf  (i.To,-  tor  oiti’eren' 


a:  " ; ; 


the  covariance  matrix  o'  the  i-th  'a  v/iUi  1 ani  the  ,‘-‘h  r-  w witt, 
t replaced  by  s is 


(3.?9) 


a ,Y=  L 


t-1  it-! 


= 0 , - 


i f r > 


, then  y 


ponents  of  u , 
~ra 


A 1 . y . A er.t!:-.  l.j’-s  : !'  (i<;u*iruti  ' twrc.s  in  e'S!;- 

t-T  ,.jt  Ps-i  ,a  * 

.1,  t-.,  an.:;  arid  the  e.-.i  ected  t'aiue  .or 


such  a tert:i  rauitipi  ied  'r;v  u.  u. 

itct  Jsy 


Vlhen  T is  fixed  an.!  N 


(3.30) 


" 717 — : i I ""ta  7-1  a 

/IT(T-]  I t=?r  a=;  ^ 


0 (0)"‘ 


has  the  same  lirnitin/'  distribution  as 


;3.31) 


where 


# 1 r* 

! = ICT  I 7- 


i’hf.  c wvoriarice  ir:atr:ix  . f the  i-!,h  and  j-th  r,.w:;.  of  (i.iit  is 
(l/(T-l)]a.,  F*  ^ . i’hus  the  limitit:,-/  distribution  of  /N(B  - B)  ii 


nonrial  with  mean  0 and  cov 


arianee  matr-ix  >10  F^  7 ' 


the  process  is  stationary  F = F and 


limiting  normal  distribution  with  mean  0 and  covariance  matrix 

E 0 . 

Limiting  distributions  when  N is  fixed  and  T ->•  oo  are  of  in- 
terest only  in  the  homogeneous  case.  For  N = 1 tiie  limiting  distri- 
bution of  i/¥(B  - B)  was  proved  in  detail  [Anderson  (1971 ),  Section 
5-5»  for  example],  and  the  method  nolds  for  any  fixed  N . Regardless 
of  the  distribution  of  if  fh®  S-re  identically  distributed 

(t  = 2,3,...),  t^ra(B  - B)  has  a limiting  normal  distribution  with 
mean  0 and  covariances  E0F^  as  T-*-®. 

It  will  be  noted  that  the  limiting  distribution  of  >/n(T-1  ) (B  - B) 
in  the  stationary  case  is  the  same  for  T fixed  and  N -+■  °<>  and  for 
N fixed  and  T ->■  «>  . We  shall  denote  this  situation  as  N and/or 
T -»  00  . Precisely  this  means  that  given  an  e > 0 there  exists  an 
Nq  and  a such  that  the  normal  distribution  differs  from  the  exact 

distribution  by  no  more  than  e if  N >.  for  any  T or  T 2. 
for  any  N . Of  course,  the  difference  is  less  than  e if  both  N >. 
and  T >.  Tq  . 
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4.  Teats  of  Hypothftses. 

4.1.  Introduction.  There  ai-e  various  r.ypotheseB  abcui  the  auto- 
regression matrices  that  one  might  want  to  test.  Because  the  asymp- 
totic distributions  of  the  estimates  of  autoregression  matrices  are 
the  same  as  the  distributions  of  estimates  of  repression  matrices  when 
the  dependent  variables  (analogrius  to  y^)  ai-e  normally  distributed 
and  the  independent  variables  (analcpous  to  ^ 'i  arc  nonstochastic, 
we  can  t-ake  over  appropriate  test  proceduresj  from  normal  regression 
theory  [Chapter  8 of  Anderson  (1958)]. 

For  many  hypotheses  there  are  choices  of  test  criteria  that  have 
the  same  limiting  distributions  under  the  respective  nuil  hypotheses. 

In  the  normal  regression  with  fixed  independent  variables  a test  pro- 
cedure is  based  on  a pxp  matrix  G which  has  a Wishart  distribution 
with  covariance  matrix  Z and  degrees  of  freedom  n and  a pxp  matrix 
H which  has  a noncentral  Wishart  distribution  with  covariance  matrix 
Z , degrees  of  freedom  k , and  a matrix-valued  noncccntrality  para- 
meter which  is  0 under  the  null  hypothesis.  Tlie  matrix  G is  pro- 


portional to  the  maximum  likelihood  estimate  Z,,  of  Z when  the  null 
hypothesis  is  true,  and  G + I!  is  proportional  to  the  maximum  like- 
lihood  estimate  Z^  of  Z when  the  null  hypothesis  ic  not  assumed 
true.  Some  criteria  are  a monotonic  function  of  the  Wilks  likelihood 
ratio  criterion  |g|/|g  + Il|  , the  Lawley-Hotelling  criterion  tr  HG  \ 
the  Filial  criterion  tr  H(g  + li)  and  the  Roy  ma;<imum  characteristic 

root  of  H G ^ (or  equivalently  of  H(G  + H)  ^).  In  this  paper  ve 
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shall  present  tr  H G~  as  the  criterion  because  it  seems  most 

analogous  to  the  corresponding  criterion  for  Markov  chains , but  any 

of  the  others  could  be  used.  All  the  criteria  have  the  same  limiting 
2 

X -distribution  when  the  null  hypothesis  is  true. 

U.2.  Specified  Autoregression  Matrices.  In  the  normal  multi- 
variate regression  model  treated  in  Chapter  8 of  Anderson  (1958) 
where  the  vector  is  from  N(Pz^,  E),a=l,  criteria 

for  testing  the  null  hypothesis  p = p depend  on  the  matrices 

(U.i)  H = (3-5*)  f (|-5*)' 

a=l 


and 


(1*.2) 


a=l 


N ^ N 

“Ixx'-Syzz'B, 

~a  ~a  ~a  ~a  c * 

a=l  a=l 


where 


.3 


tt 


uaiiii 


In  testing  the  n\ill  hypothesis  that  B(t)  = B°(t)  for  a given 
t (t  = 2,  ...,  T),  where  B°(t)  is  a completely  specified  matrix, 
the  analogue  of  ^t  1^*^^  ’ analogue  of  ^ is 

p(t)  , and  the  analogue  of  is  . The  trace  criterion  for 

testing  the  null  hypothesis  is 


(U.lt)  N tr[B(t)  - B°(t)]C^_^(0)[B(t)  - B°(t)]’E~^  . 

2 

Under  the  null  nypothesis  this  criterion  has  a limiting  X -distribu- 
2 

tion  with  p degrees  of  freedom  as  N . 

Theorem.  If  X (p^p)  has  the  distribution  N(0,  S0F  » then 

'-12  2 
tr  X F X E has  the  x -distribution  with  p degrees  of  freedom. 

Proof.  Let  F = K K* , where  K is  pxp  and  nonsingular,  and 

let  V = X K . Then 

(It. 5)  tr  X F x'e“^  = tr  V v'e~^  = tr  v'e"^  V , 


and  V is  normally  distributed  with  mean  0 ; the  covariance  between 
the  i-th  and  J-th  rows  of  V is  k'f~^K  = o^j  I . The  columns 

of  V are  independently  distributed,  each  according  to  N(0,  E)  . 

Thus  tr  V E”^  = I?_.,  V E ^ V , where  V = (v  , ...,  v ) , has  a 

-w  *w  -W  J — 1 — J — — J — —1 

2 2 

X -distribution  with  p degrees  of  freedom,  Q.E.D. 

Because  the  quantities  v1T[B(t)  - B°(t)]  are  independent  in  the 
limit  for  different  t , the  criteria  {h.h)  for  different  t are 
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1. 


It 


independent.  Hence,  to  test  the  null  hypothesis  B(t)  = B°(t)  , 

t = 2,  T,  the  sum  of  (^4. 4)  on  t from  t = 2 to  t = T can  be 

2 

used.  Under  the  null  hypothesis  this  has  a limiting  x -distribution 
2 

with  (T-2)p  degrees  of  freedom. 

If  the  autoregressive  matrices  are  the  same,  that  is, 

B(l)  = ...  B(T)  = B , one  can  teat  the  null  hypothesis  that  B is  a 
specified  pxp  matrix  B°  by  use  of  the  criterion 


(k.6)  N tr(B  - B°)  C*(0)  (B  - B°)f"^  . 

2 

As  N ->■  “>  and/or  T , this  criterion  has  a limiting  x -distribu- 
2 

tion  with  p degrees  of  freedom. 

A set  of  test  procedures  can  be  inverted  to  obtain  confidence 
regions;  a confidence  region  for  B consists  of  all  matrices  B°  such 
that  (U.6)  is  less  than  a suitable  number  from  the  x -tables. 

U.3.  EqueLLity  of  Autoregression  Matrices  given  Equality  of  Covar- 
iance Matrices.  In  case  a process  is  not  assumed  stationary  it  may  be 
of  interest  to  ask  whether  the  autoregression  coefficients  vary  over 
time.  We  shall  assume  the  distributions  of  the  disturbances  are  the 
same,  that  is,  Z^=I,  t=2,  ...,T,  for  some  covariance  matrix  Z . 
Consider  testing  the  null  hypothesis  B(t)  =B,  t=2,  ...,T,  for 

some  matrix  B , The  criterion 


2 


t 

! 

t 

I 

i 

I 

I 

I 
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(U.T)  N tr  ^ [B(t)  - B]C^  3^(0)[B(t)  - B] 

t=2  ~ 


= N tr  f B(t)  C^_^(0)  B(t)’-  B C*(0) 


has  a limiting  x^-distrihution  with  (T-2)p^  degrees  of  freedom  as 
N ->  00  when  the  null  hypothesis  is  true. 

To  Justify  the  criterion,  again  consider  the  normal  multivariate 
regression  model  where  is  from  N[p{t)  ~ •••»  N , 
t = 2,  T . Then  maximum  likelihood  estimates  are 


(i*,8) 


= I ?ta5ta' 


a=l 


ta(^^l  ~ta  5ta)  ’ 


t = 2 , . , . , T ^ 


T N 


(U.9) 


! ' nTmT  Ji  !tc.l  'Jto  - ?<*>  5ta> 


If  3(2)  = ...  =3(t)  =3  , say,  the  maximum  likelihood  estimate  of 
B is 


T K / T N \ 

~ " tL  ii  ii 


(i+.lO) 


1-1 


-ta  ~ta\ 


Z . Z , I 

ta  ~tal 


In  this  case 


(U.ll)  H = I -&1  I 2 

t=2  ~ ~ a=l 


N 


T N 


t=2  - ' t=2  a=r 


ta 
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(■QB) 


1 


has  a Wishart  distribution  with  covariance  matrix  E and  (T-2)p 

degrees  of  freedom.  The  Justification  of  this  statement  follows  from 

Anderson  (1958)  > Section  8.3,  in  a manner  similar  to  Section  8.8.  Then 

-1  ? 

the  trace  of  (U.ll)  times  E has  a x -distribution  with  degrees  of 
freedom  equal  to  T-1  times  the  number  of  components  in  times 

the  number  of  components  in  z . The  criterion  (^4.7)  is  asymptotic- 
ally  similar. 


^.4.  Independence.  The  stochastic  process  consists  of 

independent  random  vectors  if  B(t)  = 0 for  all  t in  the  inhomogeneous 
case  and  if  B = 0 in  the  homogeneous  case.  In  the  two  cases  inde- 
pendence corresponds  to  certad.n  autoregression  matrices  being  equal  to 
0 . Hence,  appropriate  criteria  are  the  special  cases  of  the  criteria 
of  Section  k,2  when  B'^(t)  = 0 and  = 0 . To  test  the  null  hypo- 
thesis B(t)  = 0 the  trace  criterion  is 


(1*.12) 


N tr  B(t)  C^_3^(0)  B(t)  £' 


To  test  B(t)  = 0,  t = 2,  ...,  T , one  can  use  the  sum  of  (U.12)  for 
t = 2,  ...,  T can  be  used.  If  E = ...  = E is  assumed  E may  be 

i “>'X> 

replaced  by  E . 

To  test  the  null  hypothesis  of  independence  in  a homogeneous 
process,  that  is,  B = 0 one  can  use  the  criterion 


(U.13) 


N tr  B C (0)  B E . 
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Note  that  the  criterion  for  testing  E{2)  = ...  B(T)  = 0 given 
Jig  = . . . = is  the  sum  of  the  criteria  for  testing  B(t)  = E , 

t = 2,  T,  for  some  matrix  B and  the  criterion  (i.l3)  for  test- 

ing that  B = 0 . 


U.3.  Test  of  a Given  Order.  The  greater  the  ordei'  of  an  auto- 
regressive process  the  further  back  in  time  is  the  dependence.  In  the 
interest  of  parsimony  an  Investigator  may  ask  whether  the  rder  of 
dependence  is  one  integer,  given  that  the  order  is  not  greater  than 
another,  which  is  larger.  In  Section  E.t  we  considered  a test  of  the 
hypcthesis  that  the  order  of  dependence  was  0 given  that  the  order 
was  not  greater  than  1.  In  general,  the  hypothesis  that  a process  is 
of  a specified  order  given  that  it  is  of  an  order  not  greater  than  a 
certain  integer  is  a nypothesis  that  the  higher-order  autoregression 
matrices  are  0 . 

To  develop  the  procedures  and  (asymptotic)  distribution  theory 
we  refer  to  the  case  of  nonstochastic  independent  variubies  (Chapter 
8 of  Anderson  (1958),  for  example).  In  the  normal  regression  model 
N(^z^,  E),  =1,  ...,  N,  partition 


(it.lt*) 


Consider  testing  the  null  hypothesis  o)  : = 0 . The  criterion  de 

pends  on  the  two  matrices 


(*+.lT) 


where 


I 


ft 


'\-l 

(1)  Jl)  1 


in'! 


N 

V 


The  regression  matrices  are  estimated  by  (3. IT)  and  E is  estimated 
by 


ih.22] 


The  criterion  then  is 
(4.23) 


N(T-r)tr  I B( i ) 


C(,i,j|r)  - I C(i  ,gjr)c(g,h|r  )c(h,J  jr)]  B(j  ) 
g,h=l~ 


k)j : 


where 


(4.24) 


' c(l,l  r) 

C(l  ,g  r ) 

C(l  ,1  r)  . 

. C(1 ,g  r) 

-1 

C(g,l  r) 

...  C(g,g  r ) 

C(g,l  r)  . 

• C(g,g  r) 

« 

2 2 

The  criterion  has  a limiting  y -distribution  with  (r-q)p  degrees 


of  freedom  under  the  null  hypothesis  as  N and/or  T . 


n¥-’- 


^ ! 
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^4.6.  Several  Processes  Icentical.  Several  populations  may  be 
sampled.  There  may  be  different  socio-economic  statuses,  different 
income  levels,  or  inviividuals  subject  to  different  treatments.  Tfie 
investigator  may  ask  wheti.er  the  several  processes  are  identical. 
Frequently  it  is  reasonable  to  treat  the  error  tenris  as  distributed 
similarly  in  the  different  populatioi: i . 

Consider  testing  the  null  hypothesis  that  the  matrices  of  auto- 
regressive coefficients  of  s first-order  homogeneous  autoregressive 
processes  with  identical  covariance  matrices  T are  equal  on  the  basis 

of  H,  observed  time  series  of  length  T from  the  h-th  process, 
h 

(b ) 

h = 1,  s . If  B ‘ is  the  estimate  of  the  matrix  for  the  h-th 

process,  B is  the  pooled  estimate  ~f  the  hypothetically  equal  matrices, 
and  E is  the  pooled  estimate  r'f  E , a criterion  for  testing  the 
null  hypothesis  is 


(1+.25) 


.r  I (B 
h=l 


(h) 


t=.' 


V 

I 

a=l 


(hi  (h) 

~t-i  ,a  d.t-1  ,a 


(B 


(h) 


E) 


'-^-1 

E 


Under  the  null  hypothesis  this  has 


a limiting 


2 

X 


-distribution  with 


. 


'i 


The  ^iustifica- 


(s  - l)p  degrees  of  freedom  as  N and/or  T . 

tion  of  the  procedure  is  similar  to  that  of  Section  it.  3. 

In  the  inhomogeneous  case  we  cein  test  the  null  hypothesis 


B^^^(t)  = ...  = B^^^(t)  for  a given  t by  use  of  (h.25)  with 


;(h), 


replaced  by  B'  Mt)  and  B replaced  by  B(t)  ; the  sioms  on  t 
are  deleted.  To  test  the  hypothesis  for  all  t = 2,  ...,  T , the 
criteria  are  summed  over  t . 


U.6.  Independence  of  Two  Subprocesses.  Suppose  y = (y  ^ y^ 

'XU  'xU 


where  and  consist  of  and  P2^Pp'*'  P2  " P^  components 


respectively.  We  ask  whether  the  two  subprocesses  {y|^^}  and 

^t  ..t 


are  independent.  Since  the  Joint  distributions  of  all  random  variables 
are  normal  with  means  0 , the  subprocesses  are  independent  if  and 
only  if 


(I+.26) 


for  all  t and  s . Let 


'^.27) 


B = 


/c'l 


h?] 

’ 2 

'hi 

I12] 

! 

\hi 

-22! 

The  two  subprocesses  (in  the  Gaussian  case)  are  independent  if 


(U.28) 


?12  " 2’  J?21  ~ 2 ’ 


29 


r 


(l*.29) 


t 


because  then 


ii.pi  , ^1^  j I 


and  are  to  be  independent,  then 

it+1  it 


(U.32) 


it 


?11  ?12\  f~ll  ? 


?21  -22!  ^ 


-22J 


’§11  §11 


L§21  §11 


B F 

-11  Ul 


?ia  E22 


■D  V 
^22  -22 j 


2 §22  ~22J 


Hence,  (i+.28)  must  hold.  Then 


(i*.33)  F = 


§11  §12^ 


§21  §22 i 


= I 

s= 

/ 


s=0  I 0 


11 

§12 

^21 

§22 

b'  0 ' 


§22  < 


^ ?u  5n  sli  ^ ?L  ?1,1-  522 

s-0  s=0 


^ §22  §21  §11  ^q§22  §22  §22 

0 s=0 


§11  2 


\2  §22  j 

implies  (t.29).  (See  Section  5.3  of  Anderson  (l97l) • ) 
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we  use 


To  test  the  hypothesis  {h.29) 


(h.3h) 


I = 


Ell 

Ei2 

/\ 

E21 

/S 

?22 

We  develop  the  test  criterion  in  anaiof.y  to  the  usual  normal  regression 


(I)'  {?)'  ' 

case.  If  X = (x  x ) has  the  distribution  Nfo,  Z), 
-a  ~a  ~a  ^ ^ ' 


a = 1,  ...  n,  where  Z is  partitioned  according  to  (t.17),  then  the 

jj  (5  ^ ^ ^ ^ E ) 

a ’ ^11.2^  ’ 


conditional  distrib 

ution 

. (1)  . 
of  X given 

~a 

-a 

where  B = E, „ E~E 

and 

z , , = z - z 

_ z 

“ -12  t22 

111. 2 til  til 

n ^ jtll 

^12 

(i4.35) 

A = 

y X X = { 

- 

a=l~“~“  \a.. 

A ^ 

\~21 

-22 

IS 


-1 


then  the  estimate  of  ^ is  A A~^  and  the  estimate  of  E 


11.2 


IS 


(l/n)(A^^  - A^2  ^2^  Ag^)  . The  trace  test  for  ^ = 0 is 


(U.36) 


^ ^12  ^2  i^ai^^ii 


~12  ~?2  ~?I ^ 


since  (l/n)  A^^  consistent  estimate  of  ~ ® when  the  null 


hypothesis  is  true,  the  criterion  can  be  simplified  to 


(i*.3T) 


^21 


To  return  to  testing  the  hypothesis  (J*.29)  for  the  autoregressive 


process,  we  can  use  the  criterion 


(U.38) 


N(T  - l)  tr  ^22  ^21  ^11  * 


2 

which  has  a limiting  X -distribution  with  degrees  of  freedom 

as  N and/or  T . 

Now  consider  testing  the  null  hypothesis  “ 2 

j;  =5;'  = 0 . As  a guide  we  look  again  to  the  usual  regression  model 

"^1  ~ ( (1)'  (2)’/  . _ , (1)'  .(2)'/ 


(1*.39) 


^^11  &12\  fill  ~12 

6 r "u  I 

-21  ^22/  \~21  ~22 


and  Z^2  ~ ^21  “ ^ ‘ estimate  of  is  distributed 

/\ 

independently  of  &22^  ’ particular,  normally  dis- 
tributed with  mean  ^^^2  covariances  constituting  ?11®?2Li  ’ 

where 


( U . Uo ) D, 


22.1 


The  test  criterion  is  an  appropriate  multiple  ot  tr  ^^2  522.1  ^21  5ll 
For  the  autoregressive  process  we  use 
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its  limiting  distribution  as  N and/or  T 


-f-  <Xt 


1 


% 

i-: 


is  a “‘distribution 
with  PjFp  degrees  of  freedom.  A criterion  to  test  the  null  hypo- 
thesis =0  is  (t.bl)  with  1 and  2 interchanged;  its  limiting 

distribution  is  aJs".  a \ -distributiou.  with  degrees  of  freedom. 

The  three  criteria  are  asymptotically  independent. 
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3.  Multiple  Decision  Prc.bleias. 

^.1.  General  Conmients.  In  a statistical  investigation  it  fre- 
quently occurs  that  several  problems  are  investigated  simultaneously 
on  the  basis  of  one  set  of  data.  The  designation  of  distinct  hypo- 
theses to  test  is  a convenient  division  of  a statistical  study.  There 
are  some  other  aspects  which  are  more  conveniently  viewed  as  making  a 
choice  or  decision  among  several  alternatives. 

3-2,  Selection  of  the  Order  of  an  Autoregressive  Process.  Suppose 
the  maximum  assumed  order  of  an  autoregressive  homogeneous  process  is 
r and  the  minimum  is  q (<r)  . It  is  desired  to  determine  whether 
the  order  is  qj  q+1 , ...,  r . A possible  procedure  is  to  test  whether 
the  order  is  r-1  against  the  alternative  that  it  is  r at  signifi- 
cance level  . If  this  null  hypothesis  is  accepted,  then  test  the 
next  mill  hypothesis  that  the  order  is  r-1  against  the  alternative  that 

it  is  r-2  at  significance  level  e , , etc.  If  the  orders 

r-1 

r-1,  r-2,  ...,  q are  accepted  in  sequence,  then  the  order  q is  de- 
termined. For  more  discussion  of  this  type  of  procedure  see  Anderson 
(l9Tl),  Section  6.U. 

3.3.  Equality  of  Regression  Matrices  over  Intervals  of  Time.  In 
the  case  of  an  inhomogeneous  autoregressive  process  it  is  possible 
that  the  autoregression  matrices  are  not  equal  over  the  entire  range 
of  time  t = ?,  ...,  T , but  are  equal  over  intervals  of  time  while 
being  unequal  between  different  intervals.  The  intervals  may  be  pre- 
assigned.  The  method  of  Section  li.3  can  be  used  to  test  simultaneously 


tWit. ■iiiifct, 
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the  hypotheses 


(5.1)  B(2)  = ...  = B(t^),  B(t^+1)  = ...  = ..., 

B(t^  T+  1)  = ...  = B(t„)  . 

~ q-1  ~ H 

The  q criteria  are  asymptotically  independent. 

In  T.  W.  Anderson  (195^)  are  examples  of  the  statistical  analysis 
of  panel  data  where  the  responses  were  discrete  and  the  Markov  chain 
served  as  a model.  The  transition  probability  matrix  expressed  the 
probability  of  a political  party  preference  at  one  interview  given 
the  preference  at  the  preceding  (monthly)  interview.  One  matrix 
fitted  the  data  for  May  to  July  (before  the  Democratic  convention), 
one  for  July  to  August  (between  the  two  conventions),  and  one  for 
August  to  October  (after  the  Republican  convention). 

If  the  autoregression  matrices  are  not  constant  for  t=  2,  ...,  T , 
there  may  be  other  patterns  to  them.  If  the  characteristic  roots 
of  B are  different  (or  if  the  elementary  divisions  are  simple), 
then  B = QAQ  ^ , where  A is  the  diagonal  matrix  of  characteris- 
tic roots  and  Q is  a matrix  of  characteristic  vectors.  In  a homo- 
geneous autoregressive  process  the  conditional  expectation  of  y^ 
given  y^_2  is 

<5'2)  <?ulh-2  • 5^’  »'t-2  ■ S'Vt  ■ 
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6.  I^irther  Discussion.  The  model  for  the  Kalman  filter  is 
usually  a generalization  of  the  inhomogeneous  process  considered  here. 
The  vector  is  an  unobservable  state  vector.  The  observed  mea- 

surement vector  is 


1 


(6.1) 


5t  = ^t  ^ 'it  ’ 


where  v is  a random  vector.  In  engineering  applications  the  matrices 

-S.0 

B('*^  / > H(t)  and  the  covariance  matrix  of  v^  are  usually  known. 
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Appendix 


This  technical  report  gives  the  mathematical  Justification  for 
assertions  in  Technical  Report  No.  2k,  elaborates  on  some  aspects,  and 
presents  some  new  methods.  This  opportunity  is  taken  to  make  some 
corrections  sund  add  an  Addendum. 

Corrections  to  Technical  Report  Mo.  2k.  Panels  and  Time  Series 
Analysis:  Markov  Chains  and  Autoregressive  Processes. 


page 

n, 

line  1 

Replace  fPij(b)  l>y  Pj^j(t)[  • 

page  l6,  line  1 
under  (3.7) 

Replace  B(l)  by  B(2)  . 

page 

17, 

line  6 

<=2CiA-i,c.U-i,c  ■ 

page 

22, 

(1.6; 

Replace  p.^^^  by  . 

page 

25, 

line  7 

Replace  B(l)  by  B(2)  . 

page 

25, 

(1.15), 

, line  2 

A. 

Replace  B by  B 1 times  • 

page 

26, 

line  1 

Delete  "given  = ...  = • 

page 

26, 

line  6 

After  "criterion"  add  "(given 

r'  \ It 

= ...  - Ir^)  * 

o ...  *-^rp  ' 

page  27, 
bottom 

line  2 

from 

1 2 
Insert  "limiting"  before  X 

Addendum:  Technical  Report  No.  kk  was  written  for  a vc'lume  to 

celebrate  tVie  Seventy-fifth  Birthday  of  Paul  F.  Lazarsfeld.  Tragically, 
Professor  hazarsfeld  died  before  the  organization  and  publication  of 
this  volume  was  completed.  The  current  technical  report  as  well  as 
the  preceding  axe  dedicated  to  his  memory. 
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